Slightly before the gps time 1384124511  (15 Nov 2023 23:01:33 UTC) the EQB1_DDS_PLL has been reconfigured to avoid glitches during the frquency update.

Just after the first SQZ lock INJ went down for few seconds but the SQZ_MAIN was already enabled and it recovered the required state (IR_AA_FMODERR_TUNED).

This night I successfully installed the pye873 package using the new version of the cmt-conda environment which has been specifically prepared for this process. I updated the main configuration of the VPM and restarted the SQZ_PI_stage_Homodyne process. The SQZ_PI_stage positioner is the one less susceptible to possible errors. That is why it was choosen for the test.

In this way we tested the migration of the pye873 process from a local directory to virgoApp.

The other two processes restart should be scheduled. I do not foresee major problems in this step. It should be as simple as restarting them. No edit to the VPM configuration is necessary.

Our goal was to update the PyE873 software in order to satisfy the request of Computing Service for the code freezing. There were two thighs to do: prepare a cmt package and use conda instead of the env.sh script. The software manages piezo-electric motors inside the AEI squeezer source and it is available here Git pye873.

The activity started on Friday 11th morning. At the beginning we moved three processes with prefix SQZ_PI_stage from the generic machine olserver113 to the one dedicated for SQZ olserver149. The processes are:

SQZ_PI_stage_2f  - beam dumper
SQZ_PI_stage_pump  - beam dumper
SQZ_PI_stage_Homodyne  - mirror

During the test the laser light was present on EQB1_HD_CAM_FF, EQB1_HD_CAM_NF and EQB2_IR_Cam for reference in case the stages moved during the activity on the software.

Only the SQZ_PI_stage_Homodyne PI stage was monitored since it controls the position of a mirror.

Record for software developers: internal servo of the PI E-873 is not stable: sometimes it moves back and forth the actuator which may cause vibrations. That's why the in-loop operation must be turned off when the mirror is parked.

The first installation of the packaged returned:

Using 'default' alias ['farmn1']
---
Installation on farmn1
---
Versions defined in '/olusers/pm_user/.config/virgo_pm_versions.txt':

  root:v5r34p051
  FFTW:v3r3p30
  VirgoPolicy:v2r6
  CSet:v2r16
  Cm:v8r12p5
  Cbf:v4r13
  Cfg:v8r06p2
  Fr:v8r34
  Fd:v8r27
  Frv:v4r31

No standard version tags found for pye873 in version control repository
Exit request sent.
Connection to farmn1 closed.

We encountered two problems:

One with the access to the drivers which we solved with the help of Computing Service. We killed the process of another user which caused problems. The SQZ_PI_stage processes in VPM remained off.

The second one is that pipython dependency is not available in the conda environment. We have sent a request for adding it into the new version of the env.

Due to the fact that pye873.py software could not connect to the drivers, the position monitoring with light was meaningless.

The work has been continued on Monday 14th.

The pye873 package has been modified and successfully installed (it still depends on the manually installed pipython dependency). Version v1r0, v1r1 and v1r2 were installed. The output log of the last installation is:

PmInstall pye873
To work in the /virgoStaging area please type your \"bawaj\" login password:
Using 'default' alias ['farmn1']
---
Installation on farmn1
---
Versions defined in '/olusers/pm_user/.config/virgo_pm_versions.txt':

  root:v5r34p051
  FFTW:v3r3p30
  VirgoPolicy:v2r6
  CSet:v2r16
  Cm:v8r12p5
  Cbf:v4r13
  Cfg:v8r06p2
  Fr:v8r34
  Fd:v8r27
  Frv:v4r31

-> pye873 v1r2 (to be installed in /virgoApp)
     Conda v2r1p0 is in /virgoApp
--------
pye873 checkout
--------
Cloning into 'v1r2'...
chgrp swmgr /virgoApp/pye873/v1r2
chgrp swmgr /virgoApp/pye873
--------
pye873 configuration
--------
Creating setup scripts.
Creating cleanup scripts.
--------
pye873 compilation
--------
#CMT---> Info: Execute action make => gmake bin=../Linux-x86_64-CL7/
#CMT---> (Makefile.header) Rebuilding ../Linux-x86_64-CL7/Linux-x86_64-CL7.make
#CMT---> (constituents.make) Rebuilding library links
#CMT---> (constituents.make) Building conda_exec.make
#CMT---> Info: Document conda_exec
#CMT---> Warning: Source file ../Linux-x86_64-CL7/bin/PyE873.py not found
#CMT---> (constituents.make) Starting conda_exec
/virgoApp/Conda/v2r1p0/scripts/generate_exec.sh: line 14: activate: No such file or directory
Using /virgo/conda/envs/cmt-conda-py39-20220317 conda environment
Creating /virgoApp/pye873/v1r2/Linux-x86_64-CL7/bin/PyE873-conda
#CMT---> (constituents.make) conda_exec done
#CMT---> all ok.
--------
Make built files read only
--------
chgrp swmgr /virgoApp/pye873/v1r2/Linux-x86_64-CL7
chgrp swmgr /virgoApp/pye873/v1r2/cmt
chgrp swmgr /virgoApp/pye873/v1r2/cfg
chgrp swmgr /virgoApp/pye873/v1r2/src
chmod 0o444 chgrp swmgr /virgoApp/pye873/v1r2/.gitlab-ci.yml
chmod 0o444 chgrp swmgr /virgoApp/pye873/v1r2/Linux-x86_64-CL7-incomplete
chmod 0o444 chgrp swmgr /virgoApp/pye873/v1r2/README.md
chmod 0o444 chgrp swmgr /virgoApp/pye873/v1r2/.gitignore
chgrp swmgr /virgoApp/pye873/v1r2/Linux-x86_64-CL7/bin
chmod 0o444 chgrp swmgr /virgoApp/pye873/v1r2/Linux-x86_64-CL7/constituents.make
chmod 0o444 chgrp swmgr /virgoApp/pye873/v1r2/Linux-x86_64-CL7/conda_exec.make
chmod 0o444 chgrp swmgr /virgoApp/pye873/v1r2/Linux-x86_64-CL7/Linux-x86_64-CL7.make
chmod 0o444 chgrp swmgr /virgoApp/pye873/v1r2/Linux-x86_64-CL7/setup.make
chmod 0o444 chgrp swmgr /virgoApp/pye873/v1r2/Linux-x86_64-CL7/make.in
chmod 0o444 chgrp swmgr /virgoApp/pye873/v1r2/Linux-x86_64-CL7/cmt_build_library_links.stamp
chmod 0o444 chgrp swmgr /virgoApp/pye873/v1r2/Linux-x86_64-CL7/conda_exec.in
chmod 0o444 chgrp swmgr /virgoApp/pye873/v1r2/Linux-x86_64-CL7/library_links.in
chmod 0o555 chgrp swmgr /virgoApp/pye873/v1r2/Linux-x86_64-CL7/bin/PyE873-conda
chmod 0o444 chgrp swmgr /virgoApp/pye873/v1r2/cmt/cleanup.sh
chmod 0o444 chgrp swmgr /virgoApp/pye873/v1r2/cmt/cleanup.csh
chmod 0o444 chgrp swmgr /virgoApp/pye873/v1r2/cmt/requirements
chmod 0o444 chgrp swmgr /virgoApp/pye873/v1r2/cmt/Makefile
chmod 0o444 chgrp swmgr /virgoApp/pye873/v1r2/cmt/setup.sh
chmod 0o444 chgrp swmgr /virgoApp/pye873/v1r2/cmt/setup.csh
chmod 0o444 chgrp swmgr /virgoApp/pye873/v1r2/cfg/PiPythonDriver.cfg
chmod 0o555 chgrp swmgr /virgoApp/pye873/v1r2/cfg/config.ini
chgrp swmgr /virgoApp/pye873/v1r2/src/lib
chmod 0o555 chgrp swmgr /virgoApp/pye873/v1r2/src/PyE873.py
chmod 0o555 chgrp swmgr /virgoApp/pye873/v1r2/src/lib/libpi_pi_gcs2.so
chmod 0o555 chgrp swmgr /virgoApp/pye873/v1r2/src/lib/libicui18n.so.30.0
chmod 0o555 chgrp swmgr /virgoApp/pye873/v1r2/src/lib/libpi_pi_gcs2.so.3.8.0
chmod 0o555 chgrp swmgr /virgoApp/pye873/v1r2/src/lib/libpi_gcs_translator.so.1.8.0
chmod 0o555 chgrp swmgr /virgoApp/pye873/v1r2/src/lib/libicudata.so.30.0
chmod 0o555 chgrp swmgr /virgoApp/pye873/v1r2/src/lib/libfbclient.so.2.5.3
chmod 0o555 chgrp swmgr /virgoApp/pye873/v1r2/src/lib/libib_util.so
chmod 0o555 chgrp swmgr /virgoApp/pye873/v1r2/src/lib/libpi_pi_gcs2-3.8.0.a
chmod 0o555 chgrp swmgr /virgoApp/pye873/v1r2/src/lib/libicuuc.so.30.0
chmod 0o555 chgrp swmgr /virgoApp/pye873/v1r2/src/lib/libfbembed.so.2.5.3
chmod 0o555 chgrp swmgr /virgoApp/pye873/v1r2/src/lib/libwxPIGraph3D.so.1.1.2
Exit request sent.
Connection to farmn1 closed.

Version v1r0 and v1r1 are not functional and should be removed. This action will be asked to the Computing.

We performed a test of the newly installed version on the SQZ_PI_stage_2f process. The other two SQZ_PI_stage processes are still running the not frozen version of the code.

The above process were installed from virgoDev into virgoApp as a preparation for the code freeze. VPM configuration was modified but the processes were not restarted to verify the successful porting.

A dedicated time-slot will be foreseen for this activity.

More details are in the git issue: https://git.ligo.org/virgo/virgoapp/Virgo/-/issues/1

and in the software repositories:

https://git.ligo.org/virgo/virgoapp/PyDdsServer

https://git.ligo.org/virgo/virgoapp/pypllserver

https://git.ligo.org/virgo/virgoapp/pythorlabs

We acquired new data for the check of FLT mirror internal resonances. We found the cavity locked so we started by detuning the IR from the resonance by 20Hz (as it is described in #60144) and we turned on the stepper motor driver of the FCIM suspension.

2023-05-12 08h12m12 UTC    bertolin process:FCEM_MotSwitch driver turned on

We proceeded with moving motor 4 and 5 back and forth in the sequence as follows:

- moved motor 4 at the speed 1 - (sorry for low accuracy in timing here)
    1367915138  12 May 2023 08:25:20 UTC  (12 May 2023 10:25:20 CEST) - ~30 seconds ago (turned on for 3 sec)

- moved motor 5 at the speed 1
    start movement at 1367915285  12 May 2023 08:27:47 UTC  (12 May 2023 10:27:47 CEST)
    stop movement at 1367915290  12 May 2023 08:27:52 UTC  (12 May 2023 10:27:52 CEST)

- moved motor 5 at the speed 1, opposite direction
    start movement 1367915421  12 May 2023 08:30:03 UTC  (12 May 2023 10:30:03 CEST)
    stop movement 1367915426  12 May 2023 08:30:08 UTC  (12 May 2023 10:30:08 CEST)

- moved motor 4 at the speed 1, opposite direction
    start movement 1367915536  12 May 2023 08:31:58 UTC  (12 May 2023 10:31:58 CEST)
    stop movement 1367915539  12 May 2023 08:32:01 UTC  (12 May 2023 10:32:01 CEST)

After the activity we disabled the motor driver:

2023-05-12 08h33m57 UTC    bertolin 'OFF [byte=0]' sent to FCIM_MotSwitch driver turned off

and put back the zero detuning to the IR at 2023-05-12 09h22m09 UTC.

Analysis of the data will follow.

Squeezer translational stages

during the squeezer alignment we have experienced communication errors between the PI_stage processes and the actuators. AEI team used the PI GUI under Windows virtual machine for the alignment.

We will take the opportunity for the debug and migration of the PI_stage process. Luca agreed to lead the migration process.

Any test involving the squeezer actuators  should be discussed with the AEI team. Before a test which risks to move randomly the actuators, all lasers must be shut down to avoid sweeping laser beams randomly over components inside the box.

MZ realignment:

       This morning we proceeded with the realignment of  the squeezer pump power MZ stabilizer   as  did one year ago (#https://logbook.virgo-gw.eu/virgo/?r=56246) and the OPO tining.

       The starting state for MZ was with a contratrast from  ~0.7 Vp-p to ~ 1.6 Vp-p around the mean value of 1.2 (see attached plot).

We acted again on the same actuator (see pictures in

OPA tuning

        Then we proceeded with the OPA. In this case, maybe due to the humidity state,we were not able to improuve the alignemnt. We decide to leave the system as it is and wait for the humidity (more) and temperature observaton in the next two weeks  to eventually act again. (Only the screws on the last steering before the OPA were moved - see picture in #https://logbook.virgo-gw.eu/virgo/?r=56246 )

Rough extimation comparing the peak to peak power:

• OPA misalignment: ~ 7%
• OPA mismatch: ~ 1%

P.S:: Before to start a problem on the remote control of the squeezer translational stages was solved (a dedicate report will follow)

The recovery of QNR system after the electrical blackout was performed on Thursday 18 May and Monday 22.

The following electronic was reswitched on:

1. Squeezer Main Laser, Squeezer CC Laser and EQB1 Subcarrier Laser
2. The 4 DDS box on the rack 5 (restarted off-on to update the IP address): g25squeeze4, g25squeeeze5, g25squeeze5 and g25squeeze7
3. The EQB1, EDB and EQB2 QPDs were restarted, both the BIAS and the Power supply
4. The 80MHz generator for the phase camera+PLL AOM was restarded via g25squeeze2 DAQ BOX
5. The 80 MHz generator used for the Main PLL was re switched on with the following parameters:
   • Frequency 80 MHz
   •  Amplitude 0 dBm
   • Frequency modulation External
   • Frequency modulation DC
   • Frequency modulation slope 44 kHz
6. The 78 MHz generator used for FMODERR loop was restored with the following parameters:
   • Frequency 78.919655  MHz
   • Amplitude pk-pk 0.9V
   • Modulation FM
   • Tuning range 600 Hz
   • Remember to switch on the output and the modulation (both the led below buttons should be on)

Part 1 - The suspended benches and mirrors are controlled again:

On thursday 18 the first action performed was the control of SQB1, SQB2, FCIM, FCEM

• SQB1:
  • we checked in the VIM the position of the DOF and we found that the setpoint in the config file were not updated in particular LC_X -3700 (last position) and LC_Y -550 (last position), we saved in the config file LC_X and we updated online LC_Y (still not saved)
  • we opened the tracking loop (SDB1 was uncontrolled)
  • we closed the angular controls
  • we closed the position controls
• SQB2: 
  • position are checked and are ok
  • we opened the tracking loops
  • closed the angular controls
  • closed te position controls
• FCIM, FCEM: 
  • with a trend we found the last position of FCIM and FCEM MAR_TX and MAR_TY when the cavity was aligned and we set it online
  • we closed the Local controls MAR_TX, MAR_TY and MAR_TZ
  • we closed the position controls of the inverted pendulum

On Monday 22 May we closed again the tracking loops of SQB1 (once SDB1 was controlled) and SQB2 once (NI was controlled). We aligned also the filter cavity maximizing the flashes in transmission.

In the meantime Henning rechecked the squeezer and the BAB was restored. A problem on SQZ Translation stages occured we could not move them anymore (up to now we can not switch from BAB to SQZ)

Part 2 - Recovery of PLLs

• Until the recovery of the INJ system we could only try to lock the CC and SC PLLs:
  • CC PLL: once restarted the CC laser it worked at the first attempt (SQZ_MAIN in LOCKED_PLL_CC _STAND_ALONE)
  • SC PLL: was not working thus we realized that the hardware was still off. Once we also switched off-on the noise eater 2 times and we could lock the PLL at the first attempt
  • MAIN PLL: once the RFC was locked we performed the following actions
    • Try to lock the PLL without succes
    • With the MUX out we checked the value of LO and RF input (looking the EQB1_MAIN_PLL_STATUS). For the LO we had a good signal for RF no. 
    • We disconnected the LO cable and we realized that was the generator. So RF cable is the beat note this means that the beat note was not present
    • We checked the Photodiode EQB1_MAIN_PLL_PD1_DC and PD2_PDC and we realized that a part of the power was missing probably the pick-off of the inj laser
    • ALS was present thus the beam in the fiber was present, thus after some investigation and with the help of Mattieu we realized that probaby the RF generator of the PLL+Phase camera AOM was not restored
    • I reconfigured the channel 1 of the g25squeeze2 DDS at 80 MHz 
    • After that we regained the missing power on the MAIN_PLL_PD1_DC and PD2_DC and the PLL locked at the first attempt

Part 3 recovery of the filter cavity

• Once the three PLLs were locked again we realigned the filter cavity and we relocked it. A huge noise was present thus we forced in the SQZ_CC_Ctrl process the filter0, the FC_Error signal based on GR_PD 
• The Filter cavity relocked but once we engaged the AA the filter cavity misaligned
• After some investigations we realized that no power on EQB1 QPDs was present. Thus we asked to Fabio to check if the QPDs were on after the ENEL Failure. He reswitched on all the QPDs and the AA worked again
• We put a Scan on PLL SC and we unlocked and relocked the cavity until we found SC flashes in transmission
• We maximized the SC flashes in transmission closing the PY_SQZ_IR loop using SC_PLL as actuator
• We closed IR_AA loop
• We manually closed the FMODERR loop and helping it putting a DC offset od 3.2V then we removed it with a120s ramp
• We performed FMODERR and we reached the state FLT_IR_AA_FMODERR_TUNED. We set 0 Hz detuning in SQZ_FLT.ini for FMODERR
• We religned SC into OPA using the SC AA loop
• We maximized the tranmission of the SC into OPA using M21 and M22 with the IR AA closed. We reached again 14.7V of IR transmission BAB+ SC, i.e. 13.6 V of SC and 1.1V of BAB
• We left for the night the filer cavity in loked IR_AA FMODERR_TUNED

Still to do:

• Update the setpoint of SQB1 MIRROR1 and 2 after the blackout
• Update the demodulation phases of SQB1 AA loop with ITF
• Realign the HD detector with SQZ beam
• The reflected beam from FLT seems clipped on EQB1 cam, to be checked
• The SQZ translation stages are still not working

Yesterday, 14th May afternoon, we replaced the temporary, additional leg (that was just an alluminum pillar) installed to unload the sorbotane ring placed in June 2022 (#https://logbook.virgo-gw.eu/virgo/?r=56211) and avoid their new collapse under the bench weight. The intervention happened without major issue, just by unscrewing the temporary pillar to lower its height and allow its removal. After that the new leg (prepared by Alessio Buggiani and mechanical workshop team) was placed by Nicola and Marco.

From the alignment side, the intervention was a relative 'soft operation' since the final bench position was so that both the green and the  bright IR beams came back very close to their previous position (we placed additional diaphragms on both GR and IR FC reflected path, to have the reference position for the recovery after the intervention). For the GR we immediately find the TEM 00 FC resonance at a above the automation locking threshold, so that was fast to recovery. The IR was recovered by acting by hand on EQB1_M3 and M5 (the piezo were saturated, so that we unloaded them and proceeded by hand). The situation before the black out is reported in attached picture. A fine alignment tuning was required.

THE PLAN was:

• fine improuvement of the ali

The aim of the Thursday shift was to lock the SC laser beam to the FC (locked with green) with a slight (20Hz) frequency detuning  with respect to the resonance condition. This was done in order to check if the mirror internal resonances (expected above 13 kHz) driven by thermal noise are visible in the IR  transmitted signal.

We used the the modified PySQZ_IR process (modification announced in #60107). To prepare the system and make measurements we took steps as follows:

1. We recovered the FLT lock with GR at 2023-05-04-10h30m37-UTC

2. We searched for the IR co-resonance at 2023-05-04-11h38m07-UTC. The SC PLL frequency to assure this condition changed with respect to the previous one by 1600 Hz from 100093497.2 Hz to 100095097.69 Hz.

3. We run the FLT automation to realign GR and IR AA. We obtained >6V of the GR transmission and >10V of the IR transmission.

4. To change the IR offset, we paused the automation and we stopped the PySQZ_IR process. This is required to change the value of the __phd_err_offset__ variable.

5. We set the __phd_err_offset__ to 20 and restarted the process. PySQZ_IR process uses the PDH error signal calibrated in frequency thus the IR offset shift from the maximum IR transmission in average by 20 Hz.

6. At this point IR transmission decreased and we left the system to acquire data starting from 2023-05-04-13h50m18-UTC.

7. At the end of the shift we set the __phd_err_offset__ back to zero and we left the system locked.

At the first glance there is not a clear evidence of mirror resonances either in the PHD error signal or in transmission from the FC. A more detailed analysis will follow.

PySQZ_IR does not need the gain sign change while switching between the actuators any more. The sign change is handled internally. The appropriate modification to the SQZ_MAIN node was introduced.

__phd_err_offset__ variable was added in the PySQZ_IR algo.py. It allows locking the IR at resonance by offsetting the PDH error signal. Access to this functionality is very limited and discouraged. Any change requires a restart of the process.

Changes are stored in dev-ir_offset branch. The merge request will happen after positive results of the testing period.

Also these processes were migrated:

• EQB1_PDU1

• EQB1_PDU2

• EQB1_PDU3

• EQB1_PDU4

• EQB1_PowerSupply

We migrated processes:

• SQZ_DDS

• SQB1_MotSwitch

• EQB1_DDS_aux0

• EQB1_DDS_Mods

• EQB1_DDS_PLL

• EQB1_ThorAct

• FCIM_MotSwitch

• FCIM_RingHeater

• FCEM_MotSwitch

• FCEM_RingHeater

After the necessary restart of the DDS'es we successfully relocked the filter cavity but the length of the cavity decreased by 0,5mm.

There are remaining processes which should be migrated:

• SQZ_PI_stage_Homodyne

• SQZ_PI_stage_pump

• SQZ_PI_stage_2f

• EQB1_PDU1

• EQB1_PDU2

• EQB1_PDU3

• EQB1_PDU4

• SQB1_Pico

• SQB1_PowerSupply

• EQB1_PowerSupply

• SQB2_Pico

• SQB2_PowerSupply

People responsible for these processes were contacted.

olserver149 virtual server was added to host processes related to the QNR.

In the VPM it got sqz_host alias.

Migration of processes will be scheduled in a proper time window in the next week.

We encountered a problem with the ThorAct process which could not start from VPM.

Manual start gave the error:

Traceback (most recent call last):
  File "/virgoDev/PyThorlabs/v2r0/scripts/./PyThorlabs.py", line 157, in <module>
    run_server(worker, init=init)
  File "/virgo/conda/envs/cmt-conda-py39-20220317/lib/python3.9/site-packages/pyserver.py", line 98, in run_server
    p = init()
  File "/virgoDev/PyThorlabs/v2r0/scripts/./PyThorlabs.py", line 148, in init
    actList = device.listAct()
  File "/virgoDev/PyThorlabs/v2r0/scripts/thorlib.py", line 13, in listAct
    pipe = os.popen(self.str_prefix + " list_actuators")
  File "/virgo/conda/envs/cmt-conda-py39-20220317/lib/python3.9/os.py", line 983, in popen
    proc = subprocess.Popen(cmd,
  File "/virgo/conda/envs/cmt-conda-py39-20220317/lib/python3.9/subprocess.py", line 951, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/virgo/conda/envs/cmt-conda-py39-20220317/lib/python3.9/subprocess.py", line 1754, in _execute_child
    self.pid = _posixsubprocess.fork_exec(
OSError: [Errno 12] Cannot allocate memory

It is suspected that memory available on the olserver113 is low.

[root@olserver113 ~]# free -h
                   total        used        free          shared    buff/cache   available
Mem:           3.8G        3.3G        130M        132M        416M        182M
Swap:          3.0G        3.0G         16K

The absence of the ThorAct triggered ad-hoc modification to the automation in order to start the measurement this morning (#59872).

On the GPS time 1364895287 SQZ_FLT unlocked because the RFC transmission dropped below the threshold (currently set to 4V) for five seconds reaching the minimum value of 3,97V. Such low transmission of the RFC is rare but lowering the automation threshold is a possible action to take in order to increase the duty cycle.

Tonight the FLT died and it did not recover automatically. We performed independent investigation of the event and we concluded that it was caused by the SQB2 horizontal control loop fault. It caused a misalignment too high to be recovered automatically.

For these reasons we modified SQZ_MAIN adding two decorators monitoring the status of the SQB1 and SQB1. The decorators warn about the event and are ready to request DOWN to SQZ_FLT node. (This line is still commented out since SQB2 control is not yet recovered)

Moreover we added a notify to the SQZ_MAIN when SQZ_FLT is in the state below GR_AA. Any state lower than this means that the FLT could not lock properly.

Marco Vardaro reported several issues spotted during long runs of the SQZ automation nodes. Below I report the problems and fixes implemented.

1. Cavity tries to lock at any cost even in hopeless cases causing log flooding and stalling other ACL processes.
Solution: assert_MINIMAL_GR_for_LOCK decorator was added to avoid exiting CONFIGURING_GR state if the GR transmission is below a threshold (currently 2. V).
The new decorator notifies by the metamedm interface about the necessity of manual cavity pre-realignment (GR lock runs immediately during the alignment when the GR transmission reaches the lock threshold).

2. assert_LFC_LOCK_FLAG_STRICT decorator does not work.
Solution: reviewed and corrected logical conditions in the decorator execution. Now it seem to work. I uncommented the decorator over the states where needed.

3. GR AA loop has a threshold higher than longitudinal lock. If it is not reached the automation does not proceed.
Solution: assert_MINIMAL_GR_for_AA decorator was added to avoid entering PREALIGNING_GR state if the GR transmission is below a threshold (currently 5. V)
The new decorator notifies by the metamedm interface about the necessity of manual cavity pre-realignment (GR AA loop runs immediately during the alignment when the GR transmission reaches the lock threshold).

4. During apparently successful relock, automation missed to engage GR AA after a brief unlock (on the 2023/04/02). This caused slow misalignment of the cavity in few hours and the unrecoverable (automatically) FLT state.
Solution: removed return True from the assert_LFC_UGF decorator. Happened an event as probable as GW detection in 2015: assert_LFC_UGF was low for one second just before the ENGAGE_GR_AA what made the automation skip this state. Improved decorator never skips states.

Among all the PLLed lasers in the QNR setup the SC exhibits the highest temperature control excursion (see: 20230402_sqz_sqzlaser.png). The oscillations correlate with the room temperature. At the begining of the first inspection we took the record of the laser crystal temperature (29,37 C) and photodiode current (1,389 A) from the control unit front panel. We started actions by fixing the tube hosting the output fibre (see before: OLWW6953.JPG and after: IMG_0006.JPG, IMG_0007.JPG)

We note that the control units of all lasers are positions in the DET EE room. SC laser control unit is in front of the air conditioning unit while MAIN and CC laser control units are on the other end of the room. Laser head of the SC laser is positioned over the door of the DET room while MAIN and CC heads are inside the squeezer enclosure.

The aim of the shift was to align the SC beam in reflection from the RR with respect the ITF OMC.

To avoid the unlocks due to the mantainance for the entire shift we locked the SQZ MAIN laser stand alone. In particular we worked with: Main PLL unlocked, CC Pll Locked (to see HD Detector Magnitude), SC PLL unlocked (to allow to scan the SC laser).

The automation state that we asked are: SQZ_MAIN in CC_PLL_LOCKED_STAND_ALONE and SQZ_FLT in DOWN.

Preliminary operations:

We took the reference of the ITF beam on SQB1 Cam0, SQB1_CAM1 and on B1p Cam.

We checked on the raw data the position of the spourious ITF beam (on Cam0 and Cam1) the 4th February at 15:35 UTC during our last shift.

Acting on the PICO M32 we put the spourious ITF beam in the same position of the 4th Feb on Cam0 and Cam1.

Looking the magnitude at 4MHz in the EQB1 HD _DIFF channel we rotated the SQB1 HWP2 waveplate to send all the SQZ beam on the ITF (+23000 steps). Then we opened the SC shutter.

UTC 7:00

We asked to the operator to put the ITF in single bounce from SR. This means both the arms misaligned and SR aligned as the last lock in Dark Fringe. On SQB1 Cam0 and Cam1, and on SDB2_B1p cameras we confirmed the presence of the ITF beam and we took a reference position. At this point we opened the SDB1 OMC shutter to have also the beam on B1s sensors.

Looking the cameras SDB2 B1s and B1p and the signals SDB2_B1_DC, SDB2_B1p_DC and SDB2_B1p_DC, we optimized the alignment of the SC beam using the position of the beam of the last shift done towards the ITF. After a first alignment moving MIRROR 1, MIRROR 2, M32 and M33.

Once the pre alignment was done we decided to scan the SC PLL frequency and align it to the OMC looking the transmitted mode on B1 photodiodes. We put a ramp on the SC PLL temperatue actuatore and we looked the OMC tranmission.

We started with a large ramp to have also the high order modes:
Temp ramp:ON [Ramp max=31500,Ramp min=29500,Period=10.]’ sent to Py_SC_PLL

UTC 9:58

We decreased the ramp to isolate and optimize the mode TM00. We maximized the peak on SDB2_B1_DC ond we checked the shape on B1p camera.

Temp ramp:ON [Ramp max=30600,Ramp min=29400,Period=5.]' sent to Py_SC_PLL

We reached a final power of 1.5mW on SDB2_B1_DC (Figure 1).
We attach the beam's position on the different cameras (SQB1 and SDB2) and the mirrors' final position (M32, M33, Mirror 1 and 2).

We rotated the HWP 1 on SQB1 and switched off the SC. We left the system with the shutter on SDB1 FI open. We will close it during the daily meeting.

SQZ_MAIN.py
assert_XXX_PLL decorators were modified to withstand short PLL unlocks. Such unlocks do not need the time costly PLL relock procedure. This effect was achieved by introducing module parameter counters.

MAIN_PLL_unlock_cnt = 10  # If you want to change it remember to do it in the decorator as well!CC_PLL_unlock_cnt = 10SC_PLL_unlock_cnt = 10

SQZ_FLT.py
On the other hand, another GR lock decorator assert_LFC_LOCK_FLAG_STRICT was added since the standard one assert_LFC_LOCK_FLAG is not effective in case of short unlocks.
The loose version is useful during GR lock and immediately after while the tight one is handy when the IR coresonance is in play.
It must be decided which states have to be guarded by the two decorators.

We implemented and tested the smooth actuator discharge (AFC_GR_M7 discharge.jpg) for all AFC actuators: FCIM, FCEM, GR and IR.

Currently the discharge is triggered at the cavity unlock in case the actuator charge exceeds the threshold. The threshld is stored in SQZ_FLT.ini.

The code was pushed to dev-main baranch of the SQZ git repository.

Last night (220921 night lock_discharge.jpg) the cavity unlocked six times and in three cases the actuators were discharged.

Turn off pllGUI every time you use it to avoid problems!

pllGUI process communication protocol with the PyPllServer was discussed and as the result, the cm_send_receive() was substituted with a ConnectOnline() to FbmPyAccess.

This is a common solution in such a case in Virgo. It requires some attention from the pllGUI users due to connection limit to the FbmPyAccess. Remember to turn off pllGUI every time you use it to avoid problems.

After the update all PyPllServer processes were restarted at 1347709101  20 Sep 2022 11:38:03 UTC  (20 Sep 2022 13:38:03 CEST).

Flooded log files were removed.

During lock in windy conditions the IR AA was not engaged due to the presence of the delay line on the EQB1. Data corresponding to the IR AA actuators are not meaningful.

Previous thresholds for loops unlock (~5*input_RMS) was too low to assure stability during bad weather conditions and it was raised arbitrarily. Below there is a comparison of the RMS deviation of the input signal into AA loops of the FLT in two different meteorological conditions. Thresholds in the automation are updated accordingly.

1346106471.0000 Sep  1 2022 22:27:33 UTC (wind speed up to 15 km/h)

1347196638.0000 Sep 14 2022 13:17:00 UTC (wind speed up to 30 km/h)

Channel name                                 Change [%]
Wind speed [km/h]        15        30        
AFC_GR_FCEM_AA_TX_INPUT  0,06028   0,2236    370.94%
AFC_GR_FCEM_AA_TY_INPUT  0,06532   0,2136    327.01%
AFC_GR_FCIM_AA_TX_INPUT  0,06672   0,198     296.76%
AFC_GR_FCIM_AA_TY_INPUT  0,06384   0,1418    222.12%

AFC_GR_M5_BPC2_X_INPUT   0,08826   0,2684    304.10%
AFC_GR_M5_BPC2_Y_INPUT   0,03165   0,07915   250.08%
AFC_GR_M7_BPC2_X_INPUT   0,08121   0,1941    239.01%
AFC_GR_M7_BPC2_Y_INPUT   0,02785   0,07771   279.03%

AFC_IR_M3_AA_X_INPUT     0,1733    0,6874    396.65%
AFC_IR_M3_AA_Y_INPUT     1,438     0,5728    39.83%
AFC_IR_M5_AA_X_INPUT     0,8258    0,2564    31.05%
AFC_IR_M5_AA_Y_INPUT     0,5087    0,3951    77.67%